Method of suppressing vr sickness, system for executing the method, and information processing device

ABSTRACT

A method includes defining a virtual space including a virtual camera. The method further includes displaying on a head-mounted device a field of view in the virtual space based on a location of the virtual camera in the virtual space. The method further includes moving the field of view by updating the image displayed on the head-mounted device. Moving of the field of view includes moving the field of view at a first speed. Moving of the field of view further includes moving the field of view at a second speed slower than the first speed.

RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/798,217 filed Oct. 30, 2017, which claims priority toJapanese Application Number 2016-213140, filed Oct. 31, 2016. Thedisclosures of all of the above-listed prior-filed applications arehereby incorporated by reference herein in their entirety.

BACKGROUND

This disclosure relates to a technology of providing virtual reality,and more specifically, to a technology of reducing visually inducedmotion sickness in virtual reality.

There is known a technology of providing virtual reality with use of ahead-mounted device (HMD). When virtual reality is provided, visuallyinduced motion sickness called virtual reality (VR) sickness may becaused. Therefore, a technology of reducing this VR sickness improves aVR experience.

Regarding a technology of reducing VR sickness, for example, in JapanesePatent No. 5869177, there is described a technology of “generating animage while suppressing the amount of information to be visuallyrecognized by a wearer of a head-mounted display when a visual-fieldimage of a virtual space to which a user is immersed is provided to theHMD” (see [Abstract]).

In WO 2015/068656 A1, there is described a technology of “generating animage to be displayed on a head-mounted display by using information ona position and a rotation acquired at a given time, and correcting thegenerated image by using information on the position and rotationupdated at a separate time” (see [Abstract]).

One cause of VR sickness is said to be the occurrence of a differencebetween a sense expected by the user based on a memory actuallyexperienced by the user and the sense actually obtained in the virtualspace (sensory conflict theory). As a result, VR sickness tends toparticularly occur when a field of view of the user in the virtual spaceis moved. Therefore, there has been proposed a technology of suppressingVR sickness by instantaneously moving the field of view of the user tohis or her intended destination in order to prevent the user fromrecognizing a movement process.

However, a method of instantaneously moving the field of view of theuser to his or her intended destination involves movements that would beimpossible in a real space, and as a result, a sense of immersion of theuser in the virtual space is reduced. Therefore, a technology forsuppressing VR sickness (visually induced motion sickness) whenproviding a virtual space, while ensuring the sense of immersion of theuser in the virtual space would improve the VR experience.

SUMMARY

This disclosure has been made in order to help solve problems such asthose described above, and an object of at least one aspect of thisdisclosure to provide a method of suppressing VR sickness (visuallyinduced motion sickness) when providing a virtual space, while ensuringa sense of immersion in the virtual space by a user. An object of atleast one aspect of this disclosure is to provide a system forsuppressing VR sickness when providing a virtual space, while ensuring asense of immersion in the virtual space by a user.

According to at least one embodiment of this disclosure, there isprovided a method to be executed by a computer to provide a virtualspace to a head-mounted device. The method includes defining a virtualspace; providing to a user of the head-mounted device a field of view inthe virtual space by displaying an image on the head-mounted device. Themethod further includes moving the field of view of the user by updatingthe image to be displayed on the head-mounted device. The moving of thefield of view includes moving the field of view at a first speed. Themoving of the field of view further includes moving the field of view ata second speed slower than the first speed.

The above-mentioned and other objects, features, aspects, and advantagesof the disclosure may be made clear from the following detaileddescription of this disclosure, which is to be understood in associationwith the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an overview of a configuration of an HMD systemof at least one embodiment of this disclosure.

FIG. 2 is a block diagram of an example of a hardware configuration of acomputer of at least one embodiment of this disclosure.

FIG. 3 is a diagram of a uvw visual-field coordinate system to be setfor an HMD of at least one embodiment of this disclosure.

FIG. 4 is a diagram of a mode of expressing a virtual space of at leastone embodiment of this disclosure.

FIG. 5 is a plan view diagram of a head of a user wearing the HMD of atleast one embodiment of this disclosure.

FIG. 6 is a diagram of a YZ cross section obtained by viewing a visuallyrecognized region from an X direction in the virtual space of at leastone embodiment of this disclosure.

FIG. 7 is a diagram of an XZ cross section obtained by viewing thevisually recognized region from a Y direction in the virtual space of atleast one embodiment of this disclosure.

FIG. 8 is a diagram of a controller of at least one embodiment of thisdisclosure.

FIG. 9 is a block diagram of a computer of at least one embodiment ofthis disclosure as a module configuration.

FIG. 10 is a flowchart of processing to be executed by the HMD system ofat least one embodiment of this disclosure.

FIG. 11A and FIG. 11B are diagrams of a state before movement of avirtual camera of at least one embodiment of this disclosure.

FIG. 12A and FIG. 12B are diagrams of a state after movement of thevirtual camera of at least one embodiment of this disclosure.

FIG. 13 is a graph of movement control of the virtual camera in responseto inputs from the controller of at least one embodiment of thisdisclosure.

FIG. 14 is a graph of movement control of the virtual camera in responseto continuous input from the controller of at least one embodiment ofthis disclosure.

FIG. 15 is a graph of movement control of the virtual camera in responseto inputs from the controller of at least one embodiment of thisdisclosure.

FIG. 16 is a flowchart of movement control of the virtual camera of atleast one embodiment of this disclosure.

FIG. 17 is a graph of movement control of the virtual camera in responseto inputs from the controller of at least one embodiment of thisdisclosure.

FIG. 18 is a flowchart of control for achieving the processing in FIG.17 of at least one embodiment of this disclosure.

FIG. 19 is a graph of movement control of the virtual camera of at leastone embodiment of this disclosure.

FIG. 20 is a graph of movement control of the virtual camera of at leastone embodiment of this disclosure.

FIG. 21 is a graph of movement control of the virtual camera of at leastone embodiment of this disclosure.

DETAILED DESCRIPTION

Now, with reference to the drawings, at least one embodiment of thisdisclosure is described in detail. In the following description, likecomponents are denoted by like reference symbols. The same applies tothe names and functions of those components. Therefore, detaileddescription of those components is not repeated.

[Configuration of HMD System]

With reference to FIG. 1, a configuration of a head-mounted device (HMD)system 100 is described. FIG. 1 is a diagram of an overview of theconfiguration of the HMD system 100 of at least one embodiment of thisdisclosure. In at least one aspect, the HMD system 100 is provided as asystem for household use or a system for professional use. An HMD mayinclude both of a so-called head-mounted display including a monitor anda head-mounted device to which a smart phone or other terminals having amonitor can be mounted.

The HMD system 100 includes an HMD 110, an HMD sensor 120, a controller160, and a computer 200. The HMD 110 includes a monitor 112 and an eyegaze sensor 140. The controller 160 may include a motion sensor 130.

In at least one aspect, the computer 200 can be connected to a network19, for example, the Internet, and can communicate to/from a server 150or other computers connected to the network 19. In another aspect, theHMD 110 may include a sensor 114 instead of the HMD sensor 120.

The HMD 110 may be worn on a head of a user to provide a virtual spaceto the user during operation. More specifically, the HMD 110 displayseach of a right-eye image and a left-eye image on the monitor 112. Wheneach eye of the user visually recognizes each image, the user mayrecognize the image as a three-dimensional image based on the parallaxof both the eyes.

The monitor 112 is achieved as, for example, a non-transmissive displaydevice. In at least one aspect, the monitor 112 is arranged on a mainbody of the HMD 110 so as to be positioned in front of both the eyes ofthe user. Therefore, when the user visually recognizes thethree-dimensional image displayed on the monitor 112, the user can beimmersed in the virtual space. According to at least one embodiment ofthis disclosure, the virtual space includes, for example, a background,objects that can be operated by the user, and menu images that can beselected by the user. According to at least one embodiment of thisdisclosure, the monitor 112 may be achieved as a liquid crystal monitoror an organic electroluminescence (EL) monitor included in a so-calledsmart phone or other information display terminals.

In at least one aspect, the monitor 112 may include a sub-monitor fordisplaying a right-eye image and a sub-monitor for displaying a left-eyeimage. In at least one aspect, the monitor 112 may be configured tointegrally display the right-eye image and the left-eye image. In thiscase, the monitor 112 includes a high-speed shutter. The high-speedshutter operates so as to enable alternate display of the right-eyeimage and the left-eye image so that only one of the eyes can recognizethe image.

In at least one aspect, the HMD 110 includes a plurality of lightsources (not shown). Each light source is achieved by, for example, alight emitting diode (LED) configured to emit an infrared ray. The HMDsensor 120 has a position tracking function for detecting the movementof the HMD 110. More specifically, the HMD sensor 120 reads a pluralityof infrared rays emitted by the HMD 110 to detect the position and theinclination of the HMD 110 in a real space.

In at least one aspect, the HMD sensor 120 may be achieved by a camera.In this case, the HMD sensor 120 may use image information of the HMD110 output from the camera to execute image analysis processing, tothereby enable detection of the position and the inclination of the HMD110.

In at least one aspect, the HMD 110 may include the sensor 114 insteadof the HMD sensor 120 as a position detector. The HMD 110 may use thesensor 114 to detect the position and the inclination of the HMD 110itself. For example, when the sensor 114 is an angular velocity sensor,a geomagnetic sensor, an acceleration sensor, or a gyrosensor, the HMD110 may use any of those sensors instead of the HMD sensor 120 to detectthe position and the inclination of the HMD 110 itself. As an example,when the sensor 114 is an angular velocity sensor, the angular velocitysensor detects over time the angular velocity about each of three axesof the HMD 110 in the real space. The HMD 110 calculates a temporalchange of the angle about each of the three axes of the HMD 110 based oneach angular velocity, and further calculates an inclination of the HMD110 based on the temporal change of the angles. Further, the HMD 110 mayinclude a transmissive display device. In this case, the transmissivedisplay device may be configured as a display device that is temporarilynon-transmissive by adjusting the transmittance of the display device.The visual-field image may include a section for presenting a real spaceon a part of the image forming the virtual space. For example, an imagephotographed by a camera mounted to the HMD 110 may be superimposed anddisplayed on a part of the visual-field image, or the real space may bevisually recognized from a part of the visual-field image by increasingthe transmittance of a part of the transmissive display device.

The eye gaze sensor 140 is configured to detect a direction (line ofsight) in which the lines of sight of the right eye and the left eye ofa user 190 are directed. The direction is detected by, for example, aknown eye tracking function. The eye gaze sensor 140 is achieved by asensor having the eye tracking function. In at least one aspect, the eyegaze sensor 140 includes a right-eye sensor and a left-eye sensor. Theeye gaze sensor 140 may be, for example, a sensor configured toirradiate the right eye and the left eye of the user 190 with infraredlight, and to receive reflection light from the cornea and the iris withrespect to the irradiation light, to thereby detect a rotational angleof each eyeball. The eye gaze sensor 140 can detect the line of sight ofthe user 190 based on each detected rotational angle.

The server 150 may transmit a program to the computer 200. In at leastone aspect, the server 150 may communicate to/from another computer 200for providing virtual reality to an HMD used by another user. Forexample, when a plurality of users play a participatory game in anamusement facility, each computer 200 communicates to/from anothercomputer 200 with a signal based on the motion of each user, to therebyenable the plurality of users to enjoy a common game in the same virtualspace.

The controller 160 is connected to the computer 200 through wired orwireless communication. The controller 160 receives input of a commandfrom the user 190 to the computer 200. In one aspect, the controller 160can be held by the user 190. In another aspect, the controller 160 canbe mounted to the body or a part of the clothes of the user 190. In atleast one aspect, the controller 160 may be configured to output atleast any one of a vibration, a sound, or light based on the signaltransmitted from the computer 200. In at least one aspect, thecontroller 160 receives from the user 190 an operation for controllingthe position and the movement of an object arranged in the virtualspace.

In at least one aspect, the motion sensor 130 is mounted on the hand ofthe user to detect the movement of the hand of the user. For example,the motion sensor 130 detects a rotational speed and the number ofrotations of the hand. The detected signal is transmitted to thecomputer 200. The motion sensor 130 is provided to, for example, theglove-type controller 160. According to at least one embodiment of thisdisclosure, for the safety in the real space, the motion sensor 130 ismounted on an object like a glove-type object that does not easily flyaway by being worn on a hand of the user 190. In at least one aspect, asensor that is not mounted on the user 190 may detect the movement ofthe hand of the user 190. For example, a signal of a camera thatphotographs the user 190 may be input to the computer 200 as a signalrepresenting the motion of the user 190. As one example, the motionsensor 130 and the computer 200 are connected to each other throughwireless communication. In the case of wireless communication, thecommunication mode is not particularly limited, and for example,Bluetooth® or other known communication methods may be used.

[Hardware Configuration]

With reference to FIG. 2, the computer 200 of at least one embodiment isdescribed. FIG. 2 is a block diagram of an example of the hardwareconfiguration of the computer 200 of at least one embodiment of thisdisclosure. The computer 200 includes, as primary components, aprocessor 10, a memory 11, a storage 12, an input/output interface 13,and a communication interface 14. Each component is connected to a bus15.

The processor 10 is configured to execute a series of commands includedin a program stored in the memory 11 or the storage 12 based on a signaltransmitted to the computer 200 or on satisfaction of a conditiondetermined in advance. In at least one aspect, the processor 10 isachieved as a central processing unit (CPU), a micro-processor unit(MPU), a field-programmable gate array (FPGA), or other devices.

The memory 11 temporarily stores programs and data. The programs areloaded from, for example, the storage 12. The data includes data inputto the computer 200 and data generated by the processor 10. In at leastone aspect, the memory 11 is achieved as a random access memory (RAM) orother volatile memories.

The storage 12 permanently stores programs and data. The storage 12 isachieved as, for example, a read-only memory (ROM), a hard disk device,a flash memory, or other non-volatile storage devices. The programsstored in the storage 12 include programs for providing a virtual spacein the HMD system 100, simulation programs, game programs, userauthentication programs, and programs for achieving communicationto/from other computers 200. The data stored in the storage 12 includesdata and objects for defining the virtual space.

In at least one aspect, the storage 12 may be achieved as a removablestorage device like a memory card. In at least one aspect, aconfiguration that uses programs and data stored in an external storagedevice may be used instead of the storage 12 built into the computer200. With such a configuration, for example, in a situation in which aplurality of HMD systems 100 are used as in an amusement facility, theprograms and the data can be collectively updated.

According to at least one embodiment of this disclosure, theinput/output interface 13 is configured to allow communication ofsignals among the HMD 110, the HMD sensor 120, and the motion sensor130. In at least one aspect, the input/output interface 13 is achievedwith use of a universal serial bus (USB), a digital visual interface(DVI), a high-definition multimedia interface (HDMI)®, or otherterminals. The input/output interface 13 is not limited to onesdescribed above.

According to at least one embodiment of this disclosure, theinput/output interface 13 may further communicate to/from the controller160. For example, the input/output interface 13 receives input of asignal output from the controller 160 and the motion sensor 130. In atleast one aspect, the input/output interface 13 transmits a commandoutput from the processor 10 to the controller 160. The commandinstructs the controller 160 to vibrate, output a sound, emit light, orthe like. When the controller 160 receives the command, the controller160 executes any one of vibration, sound output, and light emission inaccordance with the command.

The communication interface 14 is connected to the network 19 tocommunicate to/from other computers (e.g., the server 150) connected tothe network 19. In at least one aspect, the communication interface 14is achieved as, for example, a local area network (LAN), other wiredcommunication interfaces, wireless fidelity (Wi-Fi), Bluetooth®, nearfield communication (NFC), or other wireless communication interfaces.The communication interface 14 is not limited to ones described above.

In at least one aspect, the processor 10 accesses the storage 12 andloads one or more programs stored in the storage 12 to the memory 11 toexecute a series of commands included in the program. The one or moreprograms may include an operating system of the computer 200, anapplication program for providing a virtual space, and game softwarethat can be executed in the virtual space. The processor 10 transmits asignal for providing a virtual space to the HMD 110 via the input/outputinterface 13. The HMD 110 displays a video on the monitor 112 based onthe signal.

In the example in FIG. 2, the computer 200 is provided outside of theHMD 110, but in at least one aspect, the computer 200 may be built intothe HMD 110. As an example, a portable information communicationterminal (e.g., a smart phone) including the monitor 112 may function asthe computer 200.

Further, the computer 200 may be used in common among a plurality ofHMDs 110. With such a configuration, for example, the same virtual spacecan be provided to a plurality of users, and hence each user can enjoythe same application with other users in the same virtual space.

According to at least one embodiment of this disclosure, in the HMDsystem 100, a global coordinate system is set in advance. The globalcoordinate system has three reference directions (axes) that arerespectively parallel to a vertical direction, a horizontal directionorthogonal to the vertical direction, and a front-rear directionorthogonal to both of the vertical direction and the horizontaldirection in a real space. In at least one embodiment, the globalcoordinate system is one type of point-of-view coordinate system. Hence,the horizontal direction, the vertical direction (up-down direction),and the front-rear direction in the global coordinate system are definedas an x axis, a y axis, and a z axis, respectively. More specifically,the x axis of the global coordinate system is parallel to the horizontaldirection of the real space, the y axis thereof is parallel to thevertical direction of the real space, and the z axis thereof is parallelto the front-rear direction of the real space.

In at least one aspect, the HMD sensor 120 includes an infrared sensor.When the infrared sensor detects the infrared ray emitted from eachlight source of the HMD 110, the infrared sensor detects the presence ofthe HMD 110. The HMD sensor 120 further detects the position and theinclination of the HMD 110 in the real space in accordance with themovement of the user 190 wearing the HMD 110 based on the value of eachpoint (each coordinate value in the global coordinate system). In moredetail, the HMD sensor 120 can detect the temporal change of theposition and the inclination of the HMD 110 with use of each valuedetected over time.

The global coordinate system is parallel to a coordinate system of thereal space. Therefore, each inclination of the HMD 110 detected by theHMD sensor 120 corresponds to each inclination about each of the threeaxes of the HMD 110 in the global coordinate system. The HMD sensor 120sets a uvw visual-field coordinate system to the HMD 110 based on theinclination of the HMD 110 in the global coordinate system. The uvwvisual-field coordinate system set to the HMD 110 corresponds to apoint-of-view coordinate system used when the user 190 wearing the HMD110 views an object in the virtual space.

[Uvw Visual-Field Coordinate System]

With reference to FIG. 3, the uvw visual-field coordinate system isdescribed. FIG. 3 is a diagram of a uvw visual-field coordinate systemto be set for the HMD 110 of at least one embodiment of this disclosure.The HMD sensor 120 detects the position and the inclination of the HMD110 in the global coordinate system when the HMD 110 is activated. Theprocessor 10 sets the uvw visual-field coordinate system to the HMD 110based on the detected values.

In FIG. 3, the HMD 110 sets the three-dimensional uvw visual-fieldcoordinate system defining the head of the user wearing the HMD 110 as acenter (origin). More specifically, the HMD 110 sets three directionsnewly obtained by inclining the horizontal direction, the verticaldirection, and the front-rear direction (x axis, y axis, and z axis),which define the global coordinate system, about the respective axes bythe inclinations about the respective axes of the HMD 110 in the globalcoordinate system as a pitch direction (u axis), a yaw direction (vaxis), and a roll direction (w axis) of the uvw visual-field coordinatesystem in the HMD 110.

In at least one aspect, when the user 190 wearing the HMD 110 isstanding upright and is visually recognizing the front side, theprocessor 10 sets the uvw visual-field coordinate system that isparallel to the global coordinate system to the HMD 110. In this case,the horizontal direction (x axis), the vertical direction (y axis), andthe front-rear direction (z axis) of the global coordinate systemdirectly match the pitch direction (u axis), the yaw direction (v axis),and the roll direction (w axis) of the uvw visual-field coordinatesystem in the HMD 110, respectively.

After the uvw visual-field coordinate system is set to the HMD 110, theHMD sensor 120 can detect the inclination (change amount of theinclination) of the HMD 110 in the uvw visual-field coordinate systemthat is set based on the movement of the HMD 110. In this case, the HMDsensor 120 detects, as the inclination of the HMD 110, each of a pitchangle (θu), a yaw angle (θv), and a roll angle (θw) of the HMD 110 inthe uvw visual-field coordinate system. The pitch angle (θu) representsan inclination angle of the HMD 110 about the pitch direction in the uvwvisual-field coordinate system. The yaw angle (θv) represents aninclination angle of the HMD 110 about the yaw direction in the uvwvisual-field coordinate system. The roll angle (θw) represents aninclination angle of the HMD 110 about the roll direction in the uvwvisual-field coordinate system.

The HMD sensor 120 sets, to the HMD 110, the uvw visual-field coordinatesystem of the HMD 110 obtained after the movement of the HMD 110 basedon the detected inclination angle of the HMD 110. The relationshipbetween the HMD 110 and the uvw visual-field coordinate system of theHMD 110 is always constant regardless of the position and theinclination of the HMD 110. When the position and the inclination of theHMD 110 change, the position and the inclination of the uvw visual-fieldcoordinate system of the HMD 110 in the global coordinate system changein synchronization with the change of the position and the inclination.

In at least one aspect, the HMD sensor 120 may specify the position ofthe HMD 110 in the real space as a position relative to the HMD sensor120 based on the light intensity of the infrared ray or a relativepositional relationship between a plurality of points (e.g., a distancebetween the points), which is acquired based on output from the infraredsensor. Further, the processor 10 may determine the origin of the uvwvisual-field coordinate system of the HMD 110 in the real space (globalcoordinate system) based on the specified relative position.

[Virtual Space]

With reference to FIG. 4, the virtual space is further described. FIG. 4is a diagram of a mode of expressing a virtual space 2 of at least oneembodiment of this disclosure. The virtual space 2 has a structure withan entire celestial sphere shape covering a center 21 in all 360-degreedirections. In FIG. 4, in order to avoid complicated description, onlythe upper-half celestial sphere of the virtual space 2 is exemplified.Each mesh section is defined in the virtual space 2. The position ofeach mesh section is defined in advance as coordinate values in an XYZcoordinate system defined in the virtual space 2. The computer 200associates each partial image forming content (e.g., still image ormoving image) that can be developed in the virtual space 2 with eachcorresponding mesh section in the virtual space 2, to thereby provide tothe user the virtual space 2 in which a virtual space image 22 that canbe visually recognized by the user is developed.

In at least one aspect, in the virtual space 2, the XYZ coordinatesystem having the center 21 as the origin is defined. The XYZ coordinatesystem is, for example, parallel to the global coordinate system. TheXYZ coordinate system is one type of the point-of-view coordinatesystem, and hence the horizontal direction, the vertical direction(up-down direction), and the front-rear direction of the XYZ coordinatesystem are defined as an X axis, a Y axis, and a Z axis, respectively.Thus, the X axis (horizontal direction) of the XYZ coordinate system isparallel to the x axis of the global coordinate system, the Y axis(vertical direction) of the XYZ coordinate system is parallel to the yaxis of the global coordinate system, and the Z axis (front-reardirection) of the XYZ coordinate system is parallel to the z axis of theglobal coordinate system.

When the HMD 110 is activated, that is, when the HMD 110 is in aninitial state, a virtual camera 1 is arranged at the center 21 of thevirtual space 2. In at least one aspect, the processor 10 displays onthe monitor 112 of the HMD 110 an image photographed by the virtualcamera 1. In synchronization with the movement of the HMD 110 in thereal space, the virtual camera 1 similarly moves in the virtual space 2.With this, the change in position and direction of the HMD 110 in thereal space is reproduced similarly in the virtual space 2.

The uvw visual-field coordinate system is defined in the virtual camera1 similarly to the case of the HMD 110. The uvw visual-field coordinatesystem of the virtual camera in the virtual space 2 is defined to besynchronized with the uvw visual-field coordinate system of the HMD 110in the real space (global coordinate system). Therefore, when theinclination of the HMD 110 changes, the inclination of the virtualcamera 1 also changes in synchronization therewith. The virtual camera 1can also move in the virtual space 2 in synchronization with themovement of the user wearing the HMD 110 in the real space.

The processor 10 of the computer 200 defines a visually recognizedregion 23 in the virtual space 2 based on an arrangement position of thevirtual camera 1 and a reference line of sight 5. The visuallyrecognized region 23 corresponds to, of the virtual space 2, the regionthat is visually recognized by the user wearing the HMD 110.

The line of sight of the user 190 detected by the eye gaze sensor 140 isa direction in the point-of-view coordinate system obtained when theuser 190 visually recognizes an object. The uvw visual-field coordinatesystem of the HMD 110 is equal to the point-of-view coordinate systemused when the user 190 visually recognizes the monitor 112. Further, theuvw visual-field coordinate system of the virtual camera 1 issynchronized with the uvw visual-field coordinate system of the HMD 110.Therefore, in the HMD system 100 in at least one aspect, the line ofsight of the user 190 detected by the eye gaze sensor 140 can beregarded as the user's line of sight in the uvw visual-field coordinatesystem of the virtual camera 1.

[User's Line of Sight]

With reference to FIG. 5, determination of the user's line of sight isdescribed. FIG. 5 is a plan view diagram of the head of the user 190wearing the HMD 110 of at least one embodiment of this disclosure.

In at least one aspect, the eye gaze sensor 140 detects lines of sightof the right eye and the left eye of the user 190. In at least oneaspect, when the user 190 is looking at a near place, the eye gazesensor 140 detects lines of sight R1 and L1. In at least one aspect,when the user 190 is looking at a far place, the eye gaze sensor 140detects lines of sight R2 and L2. In this case, the angles formed by thelines of sight R2 and L2 with respect to the roll direction w aresmaller than the angles formed by the lines of sight R1 and L1 withrespect to the roll direction w. The eye gaze sensor 140 transmits thedetection results to the computer 200.

When the computer 200 receives the detection values of the lines ofsight R1 and L1 from the eye gaze sensor 140 as the detection results ofthe lines of sight, the computer 200 specifies a point of gaze N1 beingan intersection of both the lines of sight R1 and L1 based on thedetection values. Meanwhile, when the computer 200 receives thedetection values of the lines of sight R2 and L2 from the eye gazesensor 140, the computer 200 specifies an intersection of both the linesof sight R2 and L2 as the point of gaze. The computer 200 identifies aline of sight NO of the user 190 based on the specified point of gazeN1. The computer 200 detects, for example, an extension direction of astraight line that passes through the point of gaze N1 and a midpoint ofa straight line connecting a right eye R and a left eye L of the user190 to each other as the line of sight NO. The line of sight NO is adirection in which the user 190 actually directs his or her lines ofsight with both eyes. Further, the line of sight NO corresponds to adirection in which the user 190 actually directs his or her lines ofsight with respect to the visually recognized region 23.

In at least one aspect, the HMD system 100 may include microphones andspeakers in any part constructing the HMD system 100. When the userspeaks to the microphone, an instruction can be given to the virtualspace 2 with voice.

Further, in at least one aspect, the HMD system 100 may include atelevision broadcast reception tuner. With such a configuration, the HMDsystem 100 can display a television program in the virtual space 2.

In at least one aspect, the HMD system 100 may include a communicationcircuit for connecting to the Internet or have a verbal communicationfunction for connecting to a telephone line.

[Visually Recognized Region]

With reference to FIG. 6 and FIG. 7, the visually recognized region 23is described. FIG. 6 is a diagram of a YZ cross section obtained byviewing the visually recognized region 23 from an X direction in thevirtual space 2 of at least one embodiment of this disclosure. FIG. 7 isa diagram of an XZ cross section obtained by viewing the visuallyrecognized region 23 from a Y direction in the virtual space 2 of atleast one embodiment of this disclosure.

In FIG. 6, the visually recognized region 23 in the YZ cross sectionincludes a region 24. The region 24 is defined by the arrangementposition of the virtual camera 1, the reference line of sight 5, and theYZ cross section of the virtual space 2. The processor 10 defines arange of a polar angle α from the reference line of sight 5 serving asthe center in the virtual space as the region 24.

In FIG. 7, the visually recognized region 23 in the XZ cross sectionincludes a region 25. The region 25 is defined by the arrangementposition of the virtual camera 1, the reference line of sight 5, and theXZ cross section of the virtual space 2. The processor 10 defines arange of an azimuth R from the reference line of sight 5 serving as thecenter in the virtual space 2 as the region 25. The polar angle α andthe azimuth (are determined in accordance with the arrangement positionof the virtual camera 1 and the direction of the virtual camera 1.

In at least one aspect, the HMD system 100 causes the monitor 112 todisplay a field-of-view image 26 based on the signal from the computer200, to thereby provide the field of view in the virtual space to theuser 190. The field-of-view image 26 corresponds to a part of thevirtual space image 22, which is superimposed on the visually recognizedregion 23. When the user 190 moves the HMD 110 worn on his or her head,the virtual camera 1 is also moved in synchronization with the movement.As a result, the position of the visually recognized region 23 in thevirtual space 2 is changed. With this, the field-of-view image 26displayed on the monitor 112 is updated to an image that is superimposedon the visually recognized region 23 of the virtual space image 22 in adirection in which the user faces in the virtual space 2. The user canvisually recognize a desired direction in the virtual space 2.

In this way, the direction (inclination) of the virtual camera 1corresponds to the line of sight of the user (reference line of sight 5)in the virtual space 2, and the position in which the virtual camera 1is arranged corresponds to the point of view of the user in the virtualspace 2. Therefore, through movement (including a motion for changingthe arrangement direction and a motion for changing the direction) ofthe virtual camera 1, the image to be displayed on the monitor 112 isupdated, and the field of view (including the point of view and line-ofsight) of the user 190 is moved.

While the user 190 is wearing the HMD 110, the user 190 cannot visuallyrecognize the real world but can visually recognize only the virtualspace image 22 developed in the virtual space 2. Therefore, the HMDsystem 100 can provide a high sense of immersion in the virtual space 2to the user.

In at least one aspect, the processor 10 may move the virtual camera 1in the virtual space 2 in synchronization with the movement in the realspace of the user 190 wearing the HMD 110. In this case, the processor10 specifies an image region to be projected on the monitor 112 of theHMD 110 (that is, the visually recognized region 23 in the virtual space2) based on the position and the direction of the virtual camera 1 inthe virtual space 2.

According to at least one embodiment of this disclosure, the virtualcamera 1 includes two virtual cameras, that is, a virtual camera forproviding a right-eye image and a virtual camera for providing aleft-eye image. Further, in at least one embodiment, an appropriateparallax is set for the two virtual cameras so that the user 190 canrecognize the three-dimensional virtual space 2. In at least oneembodiment, the virtual camera 1 includes two virtual cameras, and theroll directions of the two virtual cameras are synthesized so that thegenerated roll direction (w) is adapted to the roll direction (w) of theHMD 110.

[Controller]

An example of the controller 160 is described with reference to FIG. 8.FIG. 8 is a diagram of the controller 160 of at least one embodiment ofthis disclosure.

In FIG. 8, in one aspect, the controller 160 may include a rightcontroller 800 and a left controller. The right controller 800 isoperated by the right hand of the user 190. The left controller isoperated by the left hand of the user 190. In at least one aspect, theright controller 800 and the left controller are symmetricallyconfigured as separate devices. Therefore, the user 190 can freely movehis or her right hand holding the right controller 800 and his or herleft hand holding the left controller. In another aspect, the controller160 may be an integrated controller configured to receive an operationby both hands. The right controller 800 is now described.

The right controller 800 includes a grip 30, a frame 31, and a topsurface 32. The grip 30 is configured so as to be held by the right handof the user 190. For example, the grip 30 may be held by the palm andthree fingers (middle finger, ring finger, and small finger) of theright hand of the user 190.

The grip 30 includes buttons 33 and 34 and the motion sensor 130. Thebutton 33 is arranged on a side surface of the grip 30, and isconfigured to receive an operation performed by the middle finger of theright hand. The button 34 is arranged on a front surface of the grip 30,and is configured to receive an operation performed by the index fingerof the right hand. In at least one aspect, the buttons 33 and 34 areconfigured as trigger type buttons. The motion sensor 130 is built intothe casing of the grip 30. When a motion of the user 190 can be detectedfrom the surroundings of the user 190 by a camera or other device, it isnot necessary for the grip 30 to include the motion sensor 130.

The frame 31 includes a plurality of infrared LEDs 35 arranged in acircumferential direction of the frame 31. The infrared LEDs 35 areconfigured to emit, during execution of a program using the controller160, infrared rays in accordance with progress of that program. Theinfrared rays emitted from the infrared LEDs 35 may be used to detectthe position and the posture (inclination and direction) of each of theright controller 800 and the left controller (not shown). In FIG. 8, theinfrared LEDs 35 are shown as being arranged in two rows, but the numberof arrangement rows is not limited to that in FIG. 8. The infrared LEDs35 may be arranged in one row or in three or more rows.

The top surface 32 includes buttons 36 and 37 and an analog stick 38.The buttons 36 and 37 are configured as push type buttons. The buttons36 and 37 are configured to receive an operation performed by the thumbof the right hand of the user 190. In at least one aspect, the analogstick 38 is configured to receive an operation in any direction of 360degrees from an initial position (neutral position). That operationincludes, for example, an operation for moving an object arranged in thevirtual space 2.

In at least one aspect, the right controller 800 and the left controllerinclude a battery for driving the infrared ray LEDs 35 and othermembers. The battery may be any of a primary battery and a secondarybattery. Any form of the battery may be used. For example, the batterymay be a button type or a dry cell type battery. In at least aspect, theright controller 800 and the left controller can be connected to a USBinterface of the computer 200. In this case, the right controller 800and the left controller can be supplied with power via the USBinterface.

[Control Device of HMD]

With reference to FIG. 9, the control device of the HMD 110 isdescribed. According to at least one embodiment of this disclosure, thecontrol device is achieved by the computer 200 having a knownconfiguration. FIG. 9 is a block diagram of the computer 200 of at leastone embodiment of this disclosure as a module configuration.

In FIG. 9, the computer 200 includes a display control module 220, avirtual space control module 230, a memory module 240, and acommunication control module 250. The display control module 220includes, as sub-modules, a virtual camera control module 221, afield-of-view region determining module 222, a field-of-view imagegenerating module 223, and a reference line-of-sight specifying module224. The virtual space control module 230 includes, as sub-modules, avirtual space defining module 231, a virtual object generating module232, and an operation object control module 233.

According to at least one embodiment of this disclosure, the displaycontrol module 220 and the virtual space control module 230 are achievedby the processor 10. According to at least one embodiment of thisdisclosure, a plurality of processors 10 may actuate as the displaycontrol module 220 and the virtual space control module 230. The memorymodule 240 is achieved by the memory 11 or the storage 12. Thecommunication control module 250 is achieved by the communicationinterface 14.

In at least one aspect, the display control module 220 is configured tocontrol the image display on the monitor 112 of the HMD 110.

The virtual camera control module 221 is configured to arrange thevirtual camera 1 in the virtual space 2. The virtual camera controlmodule 221 is also configured to control the arrangement position of thevirtual camera 1 and the direction (inclination) of the virtual camera 1in the virtual space 2. The field-of-view region determining module 222is configured to define the visually recognized region 23 in accordancewith the direction of the head of the user wearing the HMD 110 and thearrangement position of the virtual camera 1. The field-of-view imagegenerating module 223 is configured to generate the field-of-view image26 to be displayed on the monitor 112 based on the determined visuallyrecognized region 23.

The reference line-of-sight specifying module 224 is configured tospecify the line of sight of the user 190 based on the signal from theeye gaze sensor 140.

The virtual space control module 230 is configured to control thevirtual space 2 to be provided to the user 190. The virtual spacedefining module 231 is configured to generate virtual space datarepresenting the virtual space 2 to define the virtual space 2 in theHMD system 100.

The virtual object generating module 232 is configured to generate atarget to be arranged in the virtual space 2. Examples of the target mayinclude forests, mountains, other landscapes, and animals to be arrangedin accordance with the progression of the story of the game.

The operation object control module 233 is configured to arrange in thevirtual space 2 an operation object for receiving an operation by theuser in the virtual space 2. The user operates, for example, an objectto be arranged in the virtual space 2 by operating the operation object.In at least one aspect, examples of the operation object may include ahand object corresponding to a hand of the user wearing the HMD 110, aleg object corresponding to a leg of the user, a finger objectcorresponding to a finger of the user, and a stick object correspondingto a stick to be used by the user. When the operation object is a fingerobject, in particular, the operation object corresponds to a portion ofan axis in the direction (axial direction) indicated by that finger.

When any of the objects arranged in the virtual space 2 has collidedwith another object, the virtual space control module 230 detects thatcollision. The virtual space control module 230 can detect, for example,the timing of a given object touching another object, and performsprocessing determined in advance when the timing is detected. Thevirtual space control module 230 can detect the timing at which objectsthat are touching separate from each other, and performs processingdetermined in advance when the timing is detected. The virtual spacecontrol module 230 can also detect a state in which objects aretouching. Specifically, when the operation object and another object aretouching, the operation object control module 233 detects that theoperation object and the other object have touched, and performsprocessing determined in advance.

The memory module 240 stores data to be used for providing the virtualspace 2 to the user 190 by the computer 200. In one aspect, the memorymodule 240 stores space information 241, object information 242, anduser information 243.

The space information 241 stores one or more templates defined forproviding the virtual space 2.

The object information 242 stores content to be played in the virtualspace 2, an object to be used in that content, and information (e.g.,position information) for arranging the object in the virtual space 2.Examples of the content may include a game and content representing alandscape similar to that of the real world.

The user information 243 stores a program for causing the computer 200to function as the control device of the HMD system 100, an applicationprogram that uses each piece of content stored in the object information242, and the like.

The data and programs stored in the memory module 240 are input by theuser of the HMD 110. Alternatively, the processor 10 downloads theprograms or data from a computer (e.g., the server 150) that is managedby a business operator providing the content, to thereby store thedownloaded programs or data in the memory module 240.

The communication control module 250 may communicate to/from the server150 or other information communication devices via the network 19.

In at least one aspect, the display control module 220 and the virtualspace control module 230 may be achieved with use of, for example,Unity® provided by Unity Technologies. In another aspect, the displaycontrol module 220 and the virtual space control module 230 may also beachieved by combining the circuit elements for achieving each step ofprocessing.

The processing in the computer 200 is achieved by hardware and softwareexecuted by the processor 10. The software may be stored in advance on ahard disk or other memory module 240. The software may also be stored ona compact disc read-only memory (CD-ROM) or other computer-readablenon-volatile data recording medium, and distributed as a programproduct. The software may also be provided as a program product that canbe downloaded by an information provider connected to the Internet orother network. Such software is read from the data recording medium byan optical disc drive device or other data reading device, or isdownloaded from the server 150 or other computer via the communicationcontrol module 250 and then temporarily stored in a storage module. Thesoftware is read from the storage module by the processor 10, and isstored in a RAM in a format of an executable program. The processor 10executes that program.

The hardware constructing the computer 200 in FIG. 9 is known hardware.Therefore, at least one embodiment includes the program stored in thecomputer 200. The operations of the hardware of the computer 200 areunderstood by one of ordinary skill in the art, and hence a detaileddescription thereof is omitted here.

The data recording medium is not limited to a CD-ROM, a flexible disk(FD), and a hard disk. The data recording medium may also be anon-volatile data recording medium configured to store a program in afixed manner, for example, a magnetic tape, a cassette tape, an opticaldisc (magnetic optical (MO) disc, mini disc (MD), or digital versatiledisc (DVD)), an integrated circuit (IC) card (including a memory card),an optical card, and semiconductor memories such as a mask ROM, anelectronically programmable read-only memory (EPROM), an electronicallyerasable programmable read-only memory (EEPROM), and a flash ROM.

The term “program” referred to herein does not only include a programthat can be directly executed by the processor 10. The program may alsoinclude a program in a source program format, a compressed program, oran encrypted program, for example.

[Control Structure]

The control structure of the computer 200 of at least one embodiment isnow described with reference to FIG. 10. FIG. 10 is a flowchart ofprocessing to be executed by the HMD system 100 of at least oneembodiment of this disclosure.

With reference to FIG. 10, in Step S1010, the processor 10 of thecomputer 200 serves as the virtual space defining module 231 to specifythe virtual space image data and define the virtual space 2.

In Step S1020, the processor 10 initializes the virtual camera 1. Forexample, in a work area of the memory, the processor 10 arranges thevirtual camera 1 at the center 21 defined in advance in the virtualspace 2, and matches the direction of the virtual camera 1 with the lineof sight of the user 190.

In Step S1030, the processor 10 serves as the field-of-view imagegenerating module 223 to generate field-of-view image data fordisplaying an initial field-of-view image 26. The generatedfield-of-view image data is transmitted to the HMD 110 by thecommunication control module 250 via the field-of-view image generatingmodule 223.

In Step S1032, the monitor 112 of the HMD 110 displays the field-of-viewimage 26 based on the signal received from the computer 200. The user190 wearing the HMD 110 may recognize the virtual space 2 through visualrecognition of the field-of-view image 26.

In Step S1034, the HMD sensor 120 detects the inclination of the HMD 110based on a plurality of infrared rays emitted from the HMD 110. Thedetection result is transmitted to the computer 200 as movementdetection data.

In Step S1040, the processor 10 specifies the reference line of sight 5of the virtual camera 1 based on the inclination (movement detectiondata) of the HMD 110 input from the HMD sensor 120.

In Step S1042, the controller 160 detects an operation by the user 190in the real space. For example, in at least one aspect, the controller160 detects the fact that the analog stick has been tilted forward bythe user 190. In at least one aspect, the controller 160 detects thefact that a button has been pressed by the user 190. The controller 160transmits a detection signal representing the details of detection tothe computer 200.

In Step S1050, the processor 10 serves as the virtual camera controlmodule 221 to cause the virtual camera 1 to move in the direction of thespecified reference line of sight 5 in accordance with the detectionsignal.

In Step S1060, the processor 10 serves as the field-of-view imagegenerating module 223 to generate field-of-view image data fordisplaying the field-of-view image 26 photographed by the moved virtualcamera 1 to output the generated field-of-view image data to the HMD110.

In Step S1062, the monitor 112 of the HMD 110 updates the field-of-viewimage 26 based on the received field-of-view image data, and displaysthe updated field-of-view image. As a result, the field of view of theuser 190 in the virtual space 2 is moved. Next, update of thefield-of-view image as a result of the movement of the virtual camera 1is described with reference to FIG. 11A, FIG. 11B, FIG. 12A, and FIG.12B.

[Update of Field-of-View Image]

FIG. 11A and FIG. 11B are diagrams of a state before movement of thevirtual camera 1 of at least one embodiment of this disclosure. In FIG.11A, in at least one aspect, the user 190 wearing the HMD 110 visuallyrecognizes a field-of-view image 1100 in the virtual space 2. Thefield-of-view image 1100 includes a tree object 1110 and a mountainobject 1120. At this stage, in FIG. 11B, the virtual camera 1 isphotographing the visually recognized region 23 corresponding to thefield-of-view image 1100.

In FIG. 11A and FIG. 11B, the user 190 issues a movement command to thecontroller 160. As an example, the user 190 tilts forward the analogstick included in the controller 160. As a result, the state moves fromthe state in FIG. 11A and FIG. 11B to the state in FIG. 12A and FIG.12B.

FIG. 12A and FIG. 12B are diagrams of a state after movement of thevirtual camera 1 of at least one embodiment of this disclosure. Theprocessor 10 moves the virtual camera 1 in accordance with the detectionsignal input from the controller 160. More specifically, the processor10 specifies the reference line of sight 5 of the virtual camera 1before movement as a movement direction, and moves the virtual camera 1in the specified movement direction.

In FIG. 12B, the arrangement position of the virtual camera 1 (i.e.,point of view of the user 190) has been moved forward compared with thestate in FIG. 11B. The user 190 visually recognizes a field-of-viewimage 1200 photographed by the moved virtual camera 1. Next, movementcontrol of the virtual camera 1 is described with reference to FIG. 13to FIG. 16.

[Control Structure Relating to Movement of Virtual Camera]

FIG. 13 is a graph of movement control of the virtual camera 1 inresponse to inputs from the controller 160 in at least one embodiment ofthis disclosure. The axis of abscissa represents time, and the axis ofordinate represents a movement distance of the virtual camera 1.

As shown in FIG. 13, at a time T0, the processor 10 receives one unit ofinput from the controller 160 for moving the virtual camera 1. The oneunit of input may be a detection signal input from the controller 160during a period (one frame) in which the monitor 112 updates the image.For example, the processor 10 receives one unit of detection signal fromthe controller 160 as a result of the user 190 pressing a button on thecontroller 160 and then immediately returning the button to its originalstate, or as a result of the user 190 tilting the analog stick of thecontroller 160 in a desired direction and then immediately returning theanalog stick to its original state.

The processor 10 executes a first operation and a second operation inaccordance with reception of the one input from the controller 160.According to at least one embodiment of this disclosure, the firstoperation causes the virtual camera 1 to move rapidly, and the secondoperation causes the movement of the virtual camera 1 to stop.

In the example shown in FIG. 13, the processor 10 executes the firstoperation for rapidly moving the virtual camera 1 by a distance d duringthe period from the time T0 to a time T1 (for t1 seconds). The movementspeed of the virtual camera 1 at this time is defined as a first speed(−d/t1). Next, the processor 10 executes the second operation forstopping movement of the virtual camera 1 during the period from thetime T1 to a time T2 (for t2 seconds).

The first speed can be set to a speed at which the user feels like he orshe is teleporting. This means that the user hardly recognizes thefield-of-view image 26 during the period in which he or she is moving atthe first speed. As a result, the HMD system 100 can suppress the VRsickness of the user in response to the first operation.

The distance d is set to a distance at which the user does not lose hisor her sense of immersion in the virtual space 2. The reason for this isbecause when the distance d is too long, there is an increasedlikelihood of the user losing his or her sense of immersion in thevirtual space 2, but when the distance d is too short, the virtualcamera 1 takes a longer time to arrive at its intended destination.

At a time T3, the processor 10 again receives one input from thecontroller 160. In accordance with reception of the input, the processor10 again executes the first operation and the second operation.

As described above, the HMD system 100 of at least one embodiment ofthis disclosure can, while repeating the operations of rapidly movingthe virtual camera 1 and then stopping the virtual camera 1, move thevirtual camera 1 to the intended destination of the user 190. Thisallows the user 190 to feel as if he or she has been intermittentlymoving while his or her field of view (point of view) is repeatedlyteleported. As a result, the HMD system 100 can suppress the VR sicknessof the user 190 when moving the virtual camera 1 to the intendeddestination of the user 190. When the movement of the virtual camera 1has stopped, the user 190 visually recognizes the movement process untilhis or her intended destination. As a result, the HMD system 100 canensure a sense of immersion in the virtual space 2 by the user 190 whenthe virtual camera 1 is moved to the intended destination of the user190.

The inventor(s) of this disclosure confirmed, by using a known VRsickness measurement method, that the level of the VR sickness of a useris lower when the virtual camera is moved intermittently than when thevirtual camera is moved continuously.

According to at least one embodiment of this disclosure, the duration(t2 seconds) for which the second operation is executed can be set to belonger than the duration (t1 seconds) for which the first operation isexecuted. The reason for this is because when the duration for which thesecond operation is executed is too short, the user 190 feels as if hisor her field of view (point of view) is continuously moving.

FIG. 14 is a graph of movement control of the virtual camera 1 inresponse to continuous input from the controller 160 in at least oneembodiment of this disclosure. In FIG. 14, in at least one aspect, theprocessor 10 continuously receives from the controller 160 input formoving the virtual camera 1 during the period from the time T0 to a timeT13. For example, the processor 10 receives continuous input (detectionsignal) from the controller 160 as a result of the user 190 continuouslypressing a button on the controller 160, or as a result of the user 190continuously tilting the analog stick of the controller 160 in a desireddirection.

At the time T0, the processor 10 executes the first operation and thesecond operation in accordance with reception of the detection signalfrom the controller 160. The second operation ends at a time T11.

At the time T11, in accordance with the continued reception of thedetection signal from the controller 160, the processor 10 againexecutes the first operation and the second operation. The cycle ofmoving and stopping the virtual camera 1 is also repeated from a timeT12 to a time T14.

At the time T14, a detection signal is not being received from thecontroller 160, and hence the processor 10 does not move the virtualcamera 1. In this way, the processor 10 of one embodiment of thisdisclosure determines whether or not a detection signal from thecontroller 160 has been input at an end point of the second operation,and when the detection signal has been input, again executes the firstoperation and the second operation.

As described above, the processor 10 of at least one embodiment of thisdisclosure repeats a cycle including the first operation and the secondoperation during a period in which continuous input for moving thevirtual camera 1 is being received from the controller 160. Even basedon such movement control of the virtual camera 1, the VR sickness of theuser 190 can be suppressed while ensuring a sense of immersion in thevirtual space 2 by the user 190.

FIG. 15 is a graph of movement control of the virtual camera 1 inresponse to inputs from the controller 160 in at least one embodiment ofthis disclosure.

At the time T0, the processor 10 executes the first operation and thesecond operation in accordance with reception of the detection signalfrom the controller 160. More specifically, the processor 10 executesthe first operation during the period from the time T0 to a time T21,and executes the second operation during the period from the time T21 toa time T23.

At the time T23, the processor 10 determines whether or not thedetection signal from the controller 160 has been received duringexecution of the second operation (from time T21 to time T23). In FIG.15, the processor 10 determines that the detection signal is received ata time T22, and again executes the first operation and the secondoperation during the period from the time T23 to a time T25.

At the time T25, the processor 10 determines that the detection signalhas not been received during the second iteration of execution of thesecond operation (time T24 to time T25), and does not move the virtualcamera 1.

As described above, the processor 10 of at least one embodiment of thisdisclosure is configured to again execute a cycle including the firstoperation and the second operation when the detection signal for movingthe virtual camera 1 is received from the controller 160 duringexecution of the second operation. As a result, for example, when theuser 190 repeatedly presses a button on the controller 160, theprocessor 10 can regularly move the virtual camera 1. Even based on suchmovement control of the virtual camera 1, the VR sickness of the user190 can be suppressed while ensuring a sense of immersion in the virtualspace 2 by the user 190.

In at least one aspect, the processor 10 may be configured to againexecute a cycle including the first operation and the second operationwhen the detection signal is received during execution of the firstoperation and the second operation.

FIG. 16 is a flowchart of movement control of the virtual camera 1 of atleast one embodiment of this disclosure. Each processing step in FIG. 16can be achieved by the processor 10 of the computer 200 executing aprogram stored in the memory module 240.

In Step S1610, the processor 10 serves as the virtual space definingmodule 231 to define the virtual space 2, and to provide the virtualspace 2 to the HMD 110 worn by the user 190.

In Step S1620, based on output from the HMD sensor 120, the processor 10specifies the inclination of the HMD 110, namely, the reference line ofsight 5, as a movement direction.

In Step S1630, the processor 10 executes, in accordance with the inputfrom the controller 160 (first input), the first operation for movingthe virtual camera 1 at a first speed in the specified movementdirection.

In Step S1640, the processor 10 executes the second operation forstopping movement of the virtual camera 1.

In Step S1650, the processor 10 determines whether or not a time (t2seconds) determined in advance, in which the second operation is to beexecuted, has elapsed. In response to a determination that the timedetermined in advance has elapsed (YES in Step S1650), the processor 10ends the series of processing steps for moving the virtual camera 1. Inresponse to a determination that the time determined in advance has notelapsed (NO in Step S1650), the processor 10 advances the processing toStep S1660.

In Step S1660, the processor 10 determines whether or not a next input(second input) from the controller 160 has been received before thefirst operation and the second operation in accordance with the firstinput are complete. In response to a determination that the second inputhas been received before the first operation and the second operation inaccordance with the first input are complete (YES in Step S1660), theprocessor 10 advances the processing to Step S1670. In response to adetermination that the second input has not been received before thefirst operation and the second operation in accordance with the firstinput are complete (NO in Step S1660), the processor 10 returns theprocessing to Step S1640.

In Step S1670, the processor 10 continues the second operation forstopping movement of the virtual camera 1 until a time determined inadvance elapses. When the time determined in advance has elapsed, theprocessor 10 returns the processing to Step 31630.

In this way, the HMD system 100 of at least one embodiment of thisdisclosure can intermittently move the virtual camera 1 to the intendeddestination of the user 190 in accordance with output from thecontroller 160 operated by the user 190. This allows the user 190 tofeel as if he or she has been intermittently moving while his or herfield of view (point of view) is repeatedly teleported. As a result, theHMD system 100 can suppress the VR sickness of the user 190 when movingthe virtual camera 1 to the intended destination of the user 190, andcan ensure a sense of immersion in the virtual space 2 by the user 190.

In at least one embodiment described above, the processor 10 specifiesthe reference line of sight 5 as the movement direction. However, in atleast one aspect, the processor 10 may specify the direction in whichthe analog stick of the controller 160 has been inclined as the movementdirection.

FIG. 17 is a graph of movement control of the virtual camera 1 inresponse to inputs from the controller 160 in at least one aspect.

At the time T0, in accordance with reception of the detection signalfrom the controller 160, the processor 10 executes the first operationuntil a time T31, and then tries to execute the second operation until atime T33. In FIG. 17, the next detection signal from the controller 160is input to the processor 10 at a time T32, which is during execution ofthe second operation. In this case, at the time T32, the processor 10halts the second operation being executed, and executes a seconditeration of the first operation and the second operation.

The processor 10 again receives the next detection signal from thecontroller 160 at a time T34, which is during execution of the seconditeration of the second operation. In accordance with reception of thatnext detection signal, the processor 10 halts the second iteration ofthe second operation, and executes a third iteration of the firstoperation and the second operation.

FIG. 18 is a flowchart of control for achieving the processing shown inFIG. 17 of at least one embodiment of this disclosure. The processing inFIG. 18 is, except for the deletion of the processing of Step S1670, thesame as the processing in FIG. 16, and hence a detailed description ofthe processing that is the same is not repeated here.

In Step S1660, the processor 10 determines whether or not a next input(second input) from the controller 160 has been received before thefirst operation and the second operation in accordance with the firstinput are complete. In response to a determination that the second inputhas been received before the first operation and the second operation inaccordance with the first input are complete (YES in Step S1660), theprocessor 10 halts the first operation and the second operation inaccordance with the first input, and returns the processing to StepS1630.

As described above, for example, when the user 190 has repeatedlypressed a button on the controller 160, the processor 10 can move thevirtual camera 1 more rapidly than the processing in FIG. 15 and FIG.16.

According to at least one embodiment of this disclosure, the processor10 can determine whether or not input from the controller 160 iscontinuous input. For example, when input from the controller 160continues for a time determined in advance (e.g., time corresponding tothree frames of the monitor 112), the processor 10 can determine thatthe input is continuous input.

In response to a determination that the input from the controller 160 iscontinuous input, the processor 10 can execute the processing describedwith reference to FIG. 16, and in response to a determination that theinput from the controller 160 is not continuous input, the processor 10can execute the processing described with reference to FIG. 18. As aresult, the processor 10 can, for example, regularly move the virtualcamera 1 when the user 190 continuously presses a button on thecontroller 160, and move the virtual camera 1 more rapidly when the user190 repeatedly presses a button on the controller 160.

According to at least one embodiment of this disclosure, the processor10 is configured to cause, by rapidly moving the virtual camera 1 in thefirst operation, the user 190 to feel as if his or her field of view(point of view) has been teleported. In at least one aspect, instead ofactually rapidly moving the virtual camera 1, the processor 10 canteleport the virtual camera 1.

FIG. 19 is a graph of movement control of the virtual camera 1 in atleast one aspect. In FIG. 19, the processor 10 receives input from thecontroller 160 during the period from the time T0 to a time T43.

At the time T0, in accordance with reception of the detection signalfrom the controller 160, the processor 10 executes a first operation forteleporting the virtual camera 1 by changing the arrangement position ofthe virtual camera 1 to a position further away by a distance d. As aresult, at the frame including the time T0, the field of view (point ofview) of the user 190 is changed to a position further away by thedistance d. The processor 10 executes the first operation, and thenexecutes a second operation for stopping movement of the virtual camera1 until the time t41 (t2 seconds).

At the time T41, in accordance with the continued reception of thedetection signal from the controller 160, the processor 10 againexecutes the first operation and the second operation. Even at a timeT42, in accordance with the continued reception of the detection signalfrom the controller 160, the processor 10 again executes the firstoperation and the second operation.

The HMD system 100 of at least one embodiment of this disclosure canfurther suppress the VR sickness of the user 190 by not allowing theuser 190 to recognize the movement process of the field of view (pointof view).

In the at least one embodiment described above, the processor 10 isconfigured to execute, in accordance with input from the controller 160,the first operation for rapidly moving the virtual camera 1 and thesecond operation for stopping movement of the virtual camera 1. In atleast one aspect, instead of stopping the virtual camera 1 in the secondoperation, the processor 10 may slowly move the virtual camera 1 in thesecond operation.

FIG. 20 is a graph of movement control of the virtual camera in at leastone aspect. In FIG. 20, the processor 10 receives input from thecontroller 160 during the period from the time T0 to a time T54.

At the time T0, the processor 10 executes, in accordance with receptionof the detection signal from the controller 160, the first operation forrapidly moving the virtual camera 1 by a distance d1 during the periodfrom the time T0 to a time T51 (t1 seconds). Then, the processor 10executes the second operation for slowly moving the virtual camera 1 bya distance d2 during the period from the time T51 to a time T52 (t2seconds).

In at least one embodiment, the movement speed of the virtual camera 1in accordance with the first operation (first speed) is set to be fasterthan the movement speed of the virtual camera 1 in accordance with thesecond operation (second speed). As a result, the user 190 feels thathis or her field of view (point of view) is intermittently moving, andis less susceptible to VR sickness.

In the at least one embodiment described above, the processor 10 isconfigured to execute, in accordance with input from the controller 160,the second operation for stopping movement of the virtual camera 1 orfor slowly moving the virtual camera 1 after executing the firstoperation for rapidly moving the virtual camera 1. In at least oneaspect, the processor 10 may be configured to execute, in accordancewith input from the controller 160, the first operation after executingthe second operation.

In the at least one embodiment described above, the processor 10 isconfigured to move the virtual camera 1 at two different movement speeds(including zero) in accordance with input from the controller 160. In atleast one aspect, the processor 10 may be configured to move the virtualcamera 1 at three or more different movement speeds in accordance withinput from the controller 160.

FIG. 21 is a graph of movement control of the virtual camera in at leastone aspect. In FIG. 21, the processor 10 receives input from thecontroller 160 during the period from the time T0 to a time T63.

At the time T0, the processor 10 moves, in accordance with reception ofthe detection signal from the controller 160, the virtual camera 1 by adistance d during the period from the time T0 to a time T61. In thiscase, the processor 10 moves the virtual camera 1 while continuouslychanging the movement speed of the virtual camera 1 from high speed tolow speed.

At a time T61 and a time T62, the processor 10 repeats, in accordancewith the continued reception of the detection signal from the controller160, processing for moving the virtual camera 1 while continuouslychanging the movement speed of the virtual camera 1.

Even based on such movement control of the virtual camera 1, the VRsickness of the user 190 can be suppressed while ensuring a sense ofimmersion in the virtual space 2 by the user 190.

In the at least one embodiment described above, the processor 10 isconfigured to suppress the VR sickness of the user 190 when moving thearrangement position of the virtual camera 1, or in other words, whenmoving the point of view of the user 190. In at least one aspect, theprocessor 10 may suppress the VR sickness of the user 190 by executingthe processing described above when changing the direction of thevirtual camera 1.

A specific example of control for changing the direction of the virtualcamera 1 is now described. The processor 10 detects that the HMD 110 hasbeen inclined based on output from the HMD sensor 120. Then, inaccordance with that detection, the processor 10 changes the directionof the virtual camera 1 to an intended angle by repeating a firstoperation for changing the direction of the virtual camera 1 by a firstangular velocity, and a second operation for stopping the change in thedirection of the virtual camera 1.

With this configuration, the HMD system 100 of one embodiment of thisdisclosure intermittently moves the direction of the virtual camera 1,namely, the line of sight of the user 190. As a result, when the line ofsight of the user 190 in the virtual space 2 is moved, the HMD system100 can suppress the VR sickness of the user 190 when moving the virtualcamera 1 to the intended destination of the user 190, and can ensure asense of immersion in the virtual space 2 by the user 190.

In the at least one embodiment described above, the processor 10 isconfigured to move the virtual camera 1 in accordance with input fromthe controller 160. In at least one aspect, the processor 10 may beconfigured to move the virtual camera 1 based on position information onthe HMD 110 output from the HMD sensor 120.

In at least one aspect, the processor 10 may be configured to move thevirtual camera 1 based on a motion of the user 190 in the real space. Asan example, the processor 10 is configured to analyze image dataphotographed by a camera, and to move the virtual camera 1 during aperiod in which the user 190 is performing a motion determined inadvance (e.g., motion of moving both hands back and forth).

[Configurations]

The technical features of at least one embodiment can be summarized asfollows.

(Configuration 1)

There is provided a method to be executed by a processor 10 to provide avirtual space to an HMD 110. The method includes defining a virtualspace 2 (Step S1610). The method further includes providing to a user ofthe HMD 110 a field of view in the virtual space 2 by displaying afield-of-view image 26 on a monitor 112 of the HMD 110 (Step S1610). Themethod further includes moving a field of view (including a line ofsight and a point of view) of the user by updating the image to bedisplayed on the HMD 110. The moving of the field of view includesmoving the field of view at a first speed (Step S1630). The moving ofthe field of view further includes moving the field of view at a secondspeed slower than the first speed (Step S1640).

(Configuration 2)

In Configuration 1, the second speed includes zero. In other words, themoving of the field of view at the second speed includes stoppingmovement of the field of view (Step S1640).

(Configuration 3)

In Configuration 1 or Configuration 2, the moving of the field of viewat the second speed is executed after the moving of the field of view atthe first speed.

(Configuration 4)

In Configuration 1 to Configuration 3, the moving of the field of viewat the first speed and the moving of the field of view at the secondspeed are executed based on input from a controller 160 configured toreceive an operation by the user.

(Configuration 5)

In Configuration 4, the moving of the field of view at the first speedand the moving of the field of view at the second speed are executed inresponse to one input from the controller 160 (e.g., signal output fromthe controller 160 during one frame of the monitor 112).

(Configuration 6)

In Configuration 4, the processor 10 is configured to repeat the movingof the field of view at the first speed and the moving of the field ofview at the second speed during a period in which input for moving thefield of view continues to be received from the controller 160.

(Configuration 7)

In Configuration 4 or Configuration 5, the moving of the field of viewincludes further executing, when a second input by the controller 160 isissued before the moving of the field of view at the first speed and themoving of the field of view at the second speed, which correspond to afirst input by the controller 160, are complete, the moving of the fieldof view at the first speed and the moving of the field of view at thesecond speed corresponding to the second input after the moving of thefield of view at the first speed and the moving of the field of view atthe second speed corresponding to the first input are complete (StepS1670).

(Configuration 8)

In Configuration 4 or Configuration 5, the moving of the field of viewincludes halting, when a second input by the controller 160 is issuedbefore the moving of the field of view at the first speed and the movingof the field of view at the second speed, which correspond to a firstinput by the controller 160, are complete, the moving of the field ofview at the first speed and the moving of the field of view at thesecond speed corresponding to the first input (YES in Step S1660 of FIG.18), and then executing the moving of the field of view at the firstspeed and the moving of the field of view at the second speedcorresponding to the second input.

(Configuration 9)

In Configuration 1 to Configuration 3, the processor 10 is configured tofurther execute detecting a motion of the HMD 110. The moving of thefield of view at the first speed and the moving of the field of view atthe second speed are executed based on the detected motion of the HMD110.

(Configuration 10)

In Configuration 1 to Configuration 3, the processor 10 is configured tofurther execute detecting a motion of the user. The moving of the fieldof view at the first speed and the moving of the field of view at thesecond speed are executed based on the detected motion.

(Configuration 11)

In Configuration 1 to Configuration 10, the processor 10 is configuredto further execute arranging in the virtual space 2 a virtual camera 1configured to photograph a visual-field image 26 to be visuallyrecognized by the user. The moving of the field of view at the firstspeed includes moving the virtual camera 1 at the first speed. Themoving of the field of view at the second speed includes moving thevirtual camera 1 at the second speed.

It is to be understood that the embodiments disclosed above are merelyexamples in all aspects and in no way intended to limit this disclosure.The scope of this disclosure is defined by the appended claims and notby the above description, and it is intended that all modifications madewithin the scope and spirit equivalent to those of the appended claimsare duly included in this disclosure.

What is claimed is:
 1. A method comprising: defining a virtual space;displaying on a head-mounted device a field of view of the virtual spacebased on a point of view in the virtual space; and detecting an inputfrom a user; updating the field of view displayed on the head-mounteddevice at a first speed in response to the detected input; and updatingthe field of view displayed on the head-mounted device at a second speedin response to the detected input, wherein the first speed is differentfrom the second speed.
 2. The method according to claim 1, wherein thesecond speed is zero.
 3. The method according to claim 1, wherein thesecond speed is less than the first speed.
 4. The method according toclaim 1, wherein the updating the field of view at the first speedcomprises updating the field of view at the first speed for a firstduration, and the updating the field of view at the second speedcomprises updating the field of view at the second speed for a secondduration different from the first duration.
 5. The method according toclaim 4, wherein the second duration is longer than the first duration.6. The method according to claim 1, wherein at least one of the firstspeed or the second speed is a variable speed.
 7. The method accordingto claim 1, wherein the second speed is greater than zero.
 8. The methodaccording to claim 1, further comprising: detecting a second input fromthe user; interrupting the updating the field of view at the secondspeed in response to detecting the second input; and updating the fieldof view displayed on the head-mounted device at a third speed inresponse to detecting the second input, wherein the third speed isdifferent from the second speed.
 9. The method according to claim 1,further comprising: detecting a second input from the user during theupdating the field of view at the second speed; continuing the updatingthe field of view at the second speed after detecting the second inputuntil a direction of the updating the field of view at the second speedsatisfies a predetermined duration; and updating the field of viewdisplayed on the head-mounted device at a third speed in response todetecting the second input, wherein the third speed is different fromthe second speed.
 10. The method according to claim 1, wherein thedetecting the input comprises continuously detecting the input duringboth the updating the field of view at the first speed and the updatingthe field of view at the second speed.
 11. A system comprising: anon-transitory computer readable medium configured to store instructionsthereon; and a processor connected to the non-transitory computerreadable medium, wherein the processor is configured to execute theinstructions for: defining a virtual space; instructing a head-mounteddevice to display a field of view of the virtual space based on a pointof view in the virtual space; and detecting an input from a user;updating the field of view displayed on the head-mounted device at afirst speed in response to the detected input; and updating the field ofview displayed on the head-mounted device at a second speed in responseto the detected input, wherein the first speed is different from thesecond speed.
 12. The system according to claim 11, wherein theprocessor is configured to update the field of view using the secondspeed equal to zero.
 13. The system according to claim 11, wherein theprocessor is configured to execute the instructions for: updating thefield of view at the first speed comprises updating the field of view atthe first speed for a first duration, and updating the field of view atthe second speed comprises updating the field of view at the secondspeed for a second duration different from the first duration.
 14. Thesystem according to claim 11, wherein the processor is configured toexecute the instructions for: updating the field of view at the firstspeed comprises updating the field of view at a first variable speed; orupdating the field of view at the second speed comprises updating thefield of view at a second variable speed.
 15. The system according toclaim 11, wherein the processor is configured to execute theinstructions for: detecting a second input from the user; interruptingthe updating the field of view at the second speed in response todetecting the second input; and updating the field of view displayed onthe head-mounted device at a third speed in response to detecting thesecond input, wherein the third speed is different from the secondspeed.
 16. The system according to claim 11, wherein the processor isconfigured to execute the instructions for: detecting a second inputfrom the user during the updating the field of view at the second speed;continuing the updating the field of view at the second speed afterdetecting the second input until a direction of the updating the fieldof view at the second speed satisfies a predetermined duration; andupdating the field of view displayed on the head-mounted device at athird speed in response to detecting the second input, wherein the thirdspeed is different from the second speed.
 17. The system according toclaim 11, wherein the processor is configured to execute theinstructions for continuously detecting the input during both theupdating the field of view at the first speed and the updating the fieldof view at the second speed.
 18. A method comprising: defining a virtualspace including a virtual camera; displaying on a head-mounted device afield of view of the virtual space based on a location of the virtualcamera in the virtual space; and detecting a first input from a user;updating the field of view displayed on the head-mounted device at afirst speed in response to the detected first input; updating the fieldof view displayed on the head-mounted device at a second speed inresponse to the detected first input, wherein the first speed isdifferent from the second speed; detecting a second input from the userduring the updating the field of view at the second speed; and updatingthe field of view displayed on the head-mounted device at a third speedin response to the detected second input, wherein the third speed isdifferent from the second speed.
 19. The method according to claim 18,wherein the second speed is zero.
 20. The method according to claim 18,wherein the updating the field of view at the second speed comprisescontinuing the updating the field of view at the second speed afterdetecting the second input until a direction of the updating the fieldof view at the second speed satisfies a predetermined duration.